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Information theory this century has clarified the f9th century work of Gibbs, and has shown 
that natural units for temperature kT, defined via 1/T = dS/dE, are energy per nat of information 
uncertainty. This means that (for any system) the total thermal energy E over kT is the log- 
log derivative of multiplicity with respect to energy, and (for all b) the number of base-6 units of 
information lost about the state of the system per &-fold increase in the amount of thermal energy 
therein. For "un-inverted" {T > 0) systems, E/kT is also a temperature-averaged heat capacity, 
equaling "degrees-freedom over two" for the quadratic case. In similar units the work- free difi'erential 
heat capacity Cv/k is a "local version" of this log-log derivative, equal to bits of uncertainty gained 
per 2-fold increase in temperature. This makes Cv/k (unlike E/kT) independent of the energy zero, 
■ explaining in statistical terms its usefulness for detecting both phase changes and quadratic modes. 
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<^ • I. INTRODUCTION 

S: 

' Those conceptual approaches to a subject that offer both wider apphcabihty, and reduced algorithmic complexity, 
might be called "deep simplifications" . Minkowski's approach to special relativity via the metric equation is a classic 
Q ' example. The first impressions af "superfluous erudition" EJ have now been eclipsed, by uses for Minkowski's perspective 
O ', in developing general relativityo as well as first introductionft-|to space-tinM0Ll. 

Similarly, the communication-theory insights of ShannonB led Jaynea3 in the middle of this century to clarify 
^vq the distinction in thermal physics between "describing the dice" (i.e. physical description), and "taking the best 
J> 1 guess" (the "gambling theory" part). Thus for example, the 1st through 3rd laws of thermodynamics help "describe 
'n]" ' the dice" , while the zeroth law, as well as-,the Boltzmann (canonical) and Gibbs (grand canonical) factoES,|-a|re "dice- 
C independent" tools of statistical inferenceB. Hence we can (with increasing help from undergraduate textsHila) deepen 

understanding (including the physical intuition of computer science and biology students on matters of "code" ) by 
^ I disclosing that entropy is a measure of statistical uncertainty, expressible using information units as well as in J/K. 

^ • From this, it follows that temperature (i.e. the reciprocal of uncertainty slope 1/T = dS/dE) can be measured in 

_ units of energy per unit information (e.g. room temperature ~ ^ eV per nat). Thus heat capacities (i.e. dE/dT), 
"Y*^ ■ normally understood in units of energy per degree Kelvin, may find themselves measured in units of information 
' alone! As we will show, this is part of a larger trend in statistical physics to shift the focus from temperature (or its 
^ , reciprocal) to the physical dependence of multiplicity (or entropy) on variables conserved in the interaction between 
complex systems. 

But, what is the physical meaning of a heat capacity without reference to historical temperature units? Bits of 
what? An answer to this question (something any student might ask) does not appear to be common knowledge 
^ ' among physics teachers, so we outline an answer here. We show further why total thermal energy over kT for any 
system serves at once: (i) as the instantaneous exponent of energy in the expression for accessible state multiplicity, 
(ii) as a measure of the number of bits of (micro-canonical) uncertainty added per two-fold increase in thermal energy, 
and (iii) for quadratic systems as the number of degrees freedom over two. 



II. DESCRIBING THE DICE 



The dice of thermal physics are usually physical systems capable of accomodating thermal energy (as well as other 
quantities that may be conserved, like volume and particles) in a multiplicity of ways. This multiplicity is itself the 
key to understanding, particularly when systems are seen from the (micro-canonical) vantage point of the conserved 
quantities per sc. For example, many gases, liquids, and solids behave over some of their temperature range as though 
multiplicity (Q) is proportional to E~ , where E is thermal energy, N is the number of molecules, and ly is the number 
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of ways to store thermal energy per molecule. Such systems are called "quadratic" , since the proportionality results 
from a "sum of squares" connection between E and the state coordinates involved in storing energy. 



III. TAKING THE BEST GUESS 



Application of gambling theory to physical systems begins with the question: Where might we expect to find a 
conserved quantity X which has been randomly (as far as we know) shared between systems for so long that prior 
information about the whereabouts of X is irrelevant? In the jargon of the field, this is the same as asking: Where is 
X likely to be after equilibration! 

The science of decision-making in the presence of uncertainty (i.e. statistical inference or "gambling theory") 
suggests that the best bet is the distribution of X that can happen in the most ways, provided there is no reason to 
prefer one way over another. In the jargon of probability theory, this is the recommendation: Assume equal a priori 
probabilities, when evidence to the contrary is not available. 

For example, if systems A and B have a total energy E to share between them, then the best bet after equilibration 
will be that value of Ea = E — Eb that has the largest total multiplicity = ^Ia^b- Setting to zero the derivative of 
fl with respect to Ea, one finds that this maximum requires that j^^^j = IT^TE^^ ^^"^ hence that '^'■j^'^"^ be the same 
for both systems! These derivatives (and most others in this paper) are taken under "microcanonical constraints", 
i.e. they are partial derivatives with other extensive quantities (like volume or number of particles) held constant. 

Information theorists, of course, define the logarithm of multiplicity as uncertainty via the equation S = klnQ, and 
measure it in {nats, bits, bytes, or J/K} if k is {111^7^7^55 ,1-38 x 10"^'^}, respectively. Thus the best bet for any 
two systems sharing a conserved quantity X, in the absence of information to the contrary, is that X will rearrange 
itself between the two systems until each system's uncertainty slope (jy) has reached a common value. This is a 
quantitative version of the zeroth law of thermodynamics, based purely in the science of statistical inference, which 
applies to any systems sharing conserved quantities. 

Of course, when energy E is the quantity shared randomly between systems, the uncertainty slope ^ is the 
reciprocal temperature or coldness ^ . Hence temperature is a property which signals the propensity of a system for 
sharing of energy thermally. For example, calculating the uncertainty slope for quadratic systems from the multiplicity 
given above yields the widely useful equipartition relation: ^ — ^kT. 

When V is the quantity shared randomly between systems, the uncertainty slope ^ is the free- expansion coefficient 
equal to = = ^ at equilibriumEl. For an ideal gas, oc V'^. Solving this for ^ yields the ideal gas 

equation of state PV — NkT. When N is the quantity shared randomly, the uncertainty slope ^ is the chemical 
affinity, equal to ^ at equilibrium. From this, for example, reaction equilibrium constants may be calculated. 

This quantitative version of the zeroth law applies to all thermal systems which equilibrate, including spin systems 
(like magnets) capable of population inversions and hence negative absolute temperatures. Moreover, as a theorem 
of statistical inference not involving energy at all, it applies also to thermally unequilibrated systems sharing other 
conserved quantities (even money, for example), provided the only prior information we have is how the multiplicity 
of ways that quantity can be distributed depends on the amount of that conserved quantity to begin with! If we have 
other kinds of information, such as knowledge of a system's temperature but not its total energy, then the broader 
class of maximum entropy strategies in statistical inference (e.g. the canonical and grand ensembles) predict the 
distribution of outcomes we can expect there as well. 



IV. THERMAL ENERGY OVER KT 



A closer look shows that the statistical definition of temperature above can be rewritten as: 

1 _ dilnn) _ ( d{\iin) \ 1 



kT dE \d{\nE)fE' ^^"^ 

The quantity in curly brackets is the log-log derivative of multiplicity with respect to thermal energy. We can also 
think of this as the "instantaneous exponent" of energy in the expression of multiplicity as energy to some power, or 
as the slope of the multiplicity versus energy curve on a log-log plot. 

Rearranging the equation yields something that looks very much like the familiar equipartition theorem, except 
that the relation applies to all thermal systems under conditions of maximum ignorance (i.e. at equilibrium): 
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For quadratic systems, it is easy to see that our log-log derivative is nothing more than half the number of degrees of 
freedom (■^). For any system, however, ^ measures the instantaneous energy exponent, as well as the number of nats 
of information lost about the state of the system per e-fold increase in thermal energy of the system. The last term in 
the equality string simply notes that the value is independent of the base b of the logarithms used, provided the same 
base is used in numerator and denominator. Thus we can also think of ^ as the number of bits of information lost per 
2-fold increase in thermal energy, or more generally the number of base-6 units of information lost per &-fold increase 
in thermal energy. In our search for the meaning of heat capacity in natural units, this is our first big clue. 

Before we move on, we should also point out something that applies if our energy origin has been chosen so that 
i? ^ as T ^ 0, something we might expect for a measure of thermal energy. In terms of the no-work (e.g. constant 
volume) heat-capacity = we can write: 

with the middle equality applying only for systems NOT in a population inversion, so that absolute temperature T > 
and AT — T — Q — T . Thus when absolute temperature is positive, ^ is a heat capacity average over temperatures 
ranging between T and absolute zero. 

We've shown here that the log-log derivative of multiplicity, with respect to energy, has a simple information theo- 
retic interpretation, and is elegantly given in natural units by as well. Prom the perspective of an experimentalist, 
however, it has one glaring disadvantage: It's numeric value depends on our choice for the zero of thermal energy. 

To illustrate the problem, consider the cooling of water until it becomes ice. As water cools initially, the temperature 
drop per unit energy removed is roughly constant. One might easily say: "This looks like a quadratic system with 
about 18 degrees of freedom per molecule" , so ^ must be about 9 bits per 2-fold increase in thermal energy. Then, 
at the freezing point, the temperature stops dropping as energy continues to be removed, suggesting a quadratic 
system with nearly infinite degrees of freedom! Once all is frozen, of course, temperature continues its drop, this time 
suggesting a quadratic system with about 8 degrees of freedom per molecule, or ^ closer to 4 bits per 2-fold increase 
in thermal energy! Since kT may change little during this experiment, how can E/kT be jumping around so much? 

The answer of course is that these inference follow not by measuring total energy E, but only changes in energy. 
Moreover, in the process our preferred zero of thermal energy has been shifting about. We can see the effects of this 
more explicitly if we plot energy versus temperature for water, as shown in the lower left panel of Figure 1. The 
question then is, can we modify our estimate for the log-log derivative of thermal energy so as to reflect only data on 
temperature changes over a limited energy range? Such a quantity might allow us to probe the ways that thermal 
energy is being accomodated specifically, one energy range at a time. 



V. INSTANTANEOUS HEAT CAPACITIES 



The no-work (e.g. constant volume) instantaneous heat capacity, in natural units, can be written in terms above 
as: 



k kdT dT^^ ^ 



1 + tA' 
dT 
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Here T is absolute temperature in any units you like, and the partials are taken with work parameters (like volume) 
held constant. 

This quantity has an interesting property. If we define thermal energy E as a difference between total energy U and 
a specified "zero thermal energy" origin Uo, i.e. a.s E = U — Uo, then it is easy to see that C„ = ^ is independent 
of our choice for Uo- Thus, although ^ obviously depends on one's choice of Uo, Cy does not. 

To see what Cy actually measures, let's suppose that we have a quadratic system whose multiplicity obeys Cl = 
(U^)^, It then follows simply that f = H^^), || = 2{u-Ur) ' = ^ = it- ^hus ^ estimates not 

but ^^r^^ , where Ui is the "true origin" of thermal energy for this quadratic system. This is illustrated in Figure 
2, which plots for an ideal monatomic gas (the classical quadratic system) the same quantities plotted for water in Fig. 
1. If the system is not simply quadratic (e.g. if it has phase changes, modes of energy storage which freeze out, etc.), 
then ^ is simply a local estimate of thermal energy over kT, under the quadratic assumption. Thus ^ modifies the 
log-log derivative of multiplicity with respect to energy, by combining it with its rate of increase per e-fold change in 
temperature, to yield an estimate of ^^xT^' where Ui is a zero of thermal energy determined by assuming that the 
locally-measured log-log derivative of multiplicity is constant down to T = 0. 
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This observation also provides a different perspective on tlie mechanism by which heat capacity blows up during a 
phase change. Thermal energy over kT (or the log-log derivative of multiplicity) of course should have no singularities 
in it, since both energy and are expected to be finite for finite systems. It is thus the 2nd term in the two-term 
expression for heat capacity above, namely the temperature derivative, that provides the instability. Discontinuous 
shifts in the locally- inferred energy zero (Ui) on which heat capacity is based during a phase change thus, via this 
second term, also cause heat capacity to become singular. 

Returning to Figure 1, this raises the interesting question: Does the thermal energy of steam increase, decrease, or 
go negative, when it condenses to water? From the above, we can see that it of course decreases (perhaps even goes 
negative) if one's "zero of thermal energy" is held constant, since steam loses the latent heat of vaporization when 
it condenses, bringing it's total energy down. However, because the specific heat of water at boiling is higher than 
that for steam, energy of random motion measured with respect to our locally-inferred zero of thermal energy (e.g. 
at lOOC) actually goes up! In other words, a small part of the binding energy, liberated when water molecules fall 
into the potential well of their neighbors, goes to /em increase the energy of random motion in the condensed phase 
relative to that available to particles in uncondensed gas! 

VI. BEYOND EQUIPARTITION 

The observations above suggest that introductory texts might consider highlighting ^ per molecule in natural 
units for common substances, with no apology for the fact that it is near but not exactly half-integral in many cases. 
After all, thermal energy may not have the same access to all molecules, all of the time. This quantity nonetheless 
provides deep insight into the relationship between uncertainty and thermal energy. 

In shifting the focus from historical temperature units to the multiplicities which underlie our inferences, we can 
say that both ^ and ^ measure bits of uncertainty per 2-fold increase in energy for all physical systems, with respect 
to their respective choices for energy origin. In this sense, they represent physical quantities like degrees freedom, but 
with wider applicability. After all, degrees freedom presumes not only multiplicities that are linear with energy on a 
log-log plot (like a high-temperature Einstein solid), but it also presumes quadratic energies (i.e. energies proportional 
to a sum of squares of some "randomly-occupied" coordinates of state). The idea that "every active coordinate gets 
as we show below, even more strongly resists extension to systems with one or more entropy maxima. We begin, 
however, with a non-quadratic example of less drastic proportion. 

A. Debye Solids 

The Debye heat capacity of a solid is one case where ^ depends strongly on temperaturei. In the Debye low 
temperature limit, one has E = ^^kT, so that ^ = while % = ^ + T^^^ = 4^^^. Here 

of course, 9 is the Debye temperature related to the density and speed of sound in the solid. Note that in this limit, 
only a quarter of ^ comes from equipartition (^), the remaining three quarters from the time derivative of (in 
effect, from the unfreezing of new modes of energy accomodation). As you can see from Figure 1, such unfreezing is 
associated with a lowering of the thermal energy zero locally referenced by the heat capacity. Thus attempts to infer 
^ from the heat capacity by assuming that ^ ~ ^ yield a 4-fold overestimate of the number of degrees of freedom! 
This overestimate decreases as temperatures work themselves up to and beyond the Debye temperature 9 of the solid, 
as illustrated in Figure 1 for temperatures well above 9. There in the high temperature limit, ^ ~ ~ 3A^, as one 
expects from a classical lattice model above with ft oc E'^^ . 

B. Two-State Paramagnets 

A system more challenging to the traditional interpretation of "degrees freedom" is that for a system of N—hali- 
integral spins (i.e. a two-state paramagnet) of orientation energy e. I like it because, as Dan Schroeder saygij, "it 
forces us to think primarily in terms of entropy rather than temperature" . Begin with any system whose energy- 
storing coordinates (e.g. displacements in a potential field) in practice have an upper limit on the amount of energy 
they'll accomodate. As long as energy is low enough that no single coordinate approaches the maximum value, then 
we may well find behavior very much like that of the systems discussed above. 

However, when individual coordinate energies begin to approach their maximum (this happens quickly for two-state 
paramagnets whose coordinates accomodate but one unit of energy), things change fundamentally. In particular, there 
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will be but one way (neglecting degeneracies) for the system to store the maximum amount of energy (namely when 
each of the coordinates is fully energized). With but one way to accomodate either minimum energy or maximum 
energy, multiplicities will approach 1 at both endpoints of the continuum. Since large systems may have many ways to 
store intermediate amounts of energy, multiplicity (and entropy) as a function of system energy will have a maximum 
(or maxima) somewhere between. At such maxima, ^ = will be zero, while on their high-energy side ^ = 
will be negative (signaling "population-inverted" states not accessible by thermal contact with reservoirs at positive 
absolute temperature). 

Taylor-expanding to second order about the energy (-Egmax) of such multiplicity maxima gives S{E) in the 
neighborhood as S{Esnia.x) + ^A{E — Esme^x)^ , where A = (j^j < 0. Hence for energies near -Eg max, 

" ^ ' E—Eq max 

^ = ~ A{E — Esrnax)- Thus deviations from i?s max are negative at positive absolute temperature, and (for 
small deviations at least) are proportional to reciprocal temperature /? = as illustrated in Figure 3. Of course, 
this law was discovered by Pierre Curie in experimental study of magnetization, and bears his name. 

For two-state paramagnets, A — — -rr-j- and Esmax = where Eb is the energy of alignment per spin (magnetic 

moment times magnetic field strength). Thus for E near Esmax, ^ — A^e^/?^ > 0, while ~ < 0. 

Since /3 for these systems may be positive or negative, ^ will be negative for some /3 values regardless of our choice of 
the thermal energy zero! Although negative "degrees freedom" may cause discomfort for some, a negative value for ^ 
should disturb no one since, to paraphrase a related comment by Schroedertj, there's no law of physics guaranteeing 
that there will not be fewer ways to distribute energy, as more energy is added. 

In fact, as we now know, temperature and reciprocal temperature are sigiely different forms for the Lagrange 
multiplier that characterizes a system's willingness to share thermal energyQfl. Systems, like these spin systems, 
capable of taking on (and sharing energy from) negative absolute temperature states show clearly that for them 
reciprocal temperature has more fundamental significance, and that the "absolute zeros" of temperature (approached 
from negative or positive directions) are indeed at opposite ends of a continuumllj. But if reciprocal temperature 
is more fundamental, our instantaneous no-work heat capacity should be no less simply connected to the log-log 
derivative via reciprocal temperature. Rearrangement of the equation above shows that indeed this is the case. 
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Had we historically adopted as our measure of "willingness to share energy" some other power of the uncertainty 
slope, say 7 = where a ^ — 1, the instantaneous heat capacity would have remained proportional to the log- log 
derivative of multiplicity with respect to that measure as well. 

This version of the relation now lets us simplify our perspective on entropy maxima. In the continuum (Stirling) 
approximation for spin system accessible states VI, and measuring energy from the "low-energy side", equation (|l|) 
yields E ~ Nel{l + e^) where x = e/kT, so that ^ = Nx/{1 + e^) and ^ = [x/{l + e^)fe^N. Here of course, 
e/kT takes on positive and negative values, ranging from around -|-ln(A'^) to — ln(A'^) respectively for orientation 
energies E with allowed values from to A^e. As you can see from the plot in Fig. 3, again ^ overestimates 
at low temperatures (high values of e/kT), although the estimate becomes exact when e/kT decreases to around 
a; = = 1.279 (the solution of e^^ = x — \). After this ^ underestimates which begins to decrease as x 
decreases and T increases from this point. 

All of this switches again when -gj; passes through zero, since ^ remains positive while the change in uncertainty 
per e-fold increase in energy (^ = ^) becomes negative since uncertainty about the system state decreases with added 
energy past this point. Also, of course, average heat capacity goes to zero and no longer equals since the average 
must be obtained piecewise when temperature (unlike reciprocal temperature) breaches the discontinuity from plus 
infinity to minus infinity. Thus in addition to information units for heat capacity, we gain from this approach a way 
to visualize the limits of equipartition, and minimize consternation over negative degrees of freedom (e.g. for spin 
and virial systems) as well. 



VII. SUMMARY 



In short, we've looked here at natural (as distinct from historical) units for the common thermodynamic quantities, 
so that we might explore the possibility that common uses of T and 1/T, as measures of "willingness to share thermal 
energy" , have inherited their present emphasis partly because they predate our present understanding of multiplicity 
(the W m. S = k\n.W on Josiah Willard Gibb's tombstone). Heat capacity is a particularly knotty concept in this 
regard, since for most of us it has always been a change in energy "per degree Kelvin" . In fundamental units, if heat 
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capacity has any dimensions at all they are those of the basc-& information units, since "change in energy per unit 
change in energy per bit" leaves us with nothing but bits in the bargain. 

We point out a simple interpretation for the result, namely that thermal energy E, divided by temperature T = 
^ = f3~^, is fundamentally ^ = E^, i.e. the log-log derivative of multiplicity with respect to energy (e.g. a 
measure of the bits of uncertainty increase per two- fold increase in energy). The instantaneous no- work heat capacity 
in this context is ^ = T|y = — /3§|, an estimate of ^ with an energy-zero inferred from the "local slope" of the 
log-log plot. We show that these two quantities bear a simple relationship to each other, regardless of the variable 
(e.g. T or chosen to keep track of a system's willingness to share energy thermally. 

The former of the quantities, namely plays the role of "degrees freedom over two" in quadratic systems, but 
is dependent on the energy zero, and regardless can take on negative values in systems with entropy maxima. Its 
limitations are those of the concept of equipartition itself. The latter quantity, namely provides deep insight into 
the ways a system accomodates new thermal energy. Because these quantities are defined in terms of state multiplicity 
and the conserved variable being shared (in this case energy), and relatively independent of the form chosen for the 
Lagrange multiplier in the problem (e.g. temperature in historical units), their analogs in problems that involve the 
sharing of other conserved quantities (e.g. volume, particles, or even dollars) may be easier to recognize and put to 
use as well. 
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FIG. 1. Plot of E versus kT (lower left), Cy/k versus E/kT (upper right), and the entropy plots whose slopes associate 
therewith (see text), for a steam, water, and Debye-model ice system {Toebye = 333K assuming sound speed near 3500[m/s]) 
at 1 atmosphere. This plot illustrates the use of physical (as distinct from historical) temperature units, and of dimcnsionless 
units (base-b information units per b-fold increase in T or E) for specific heat Cv/k and multiplicity's energy-exponent E/kT. 
All quantities axe "per molecule" values. Note in particular how the quadratic model "energy of thermal motion" for water 
increases as water condenses from a gas to a liquid. Also note how equipartition applies only far from phase changes and "freeze 
out" zones, and then only if a fictituous zero for the thermal energy is chosen. 

FIG. 2. Plot of E versus kT (lower left), Cv/k versus E/kT (upper right), and the entropy plots whose slopes associate 
therewith (see text), for an ideal (Sakur- Tetrode) monatomic 6000-atom argon gas at 1 atmosphere. This figure illustrates the 
form taken by the plots introduced in Fig. 1, for the case of an ideal "quadratic system". All quantities are "per atom" values. 
This system behaves like many physical systems (including low-density gases and Dulong-Petit metals), but only when well 
away from temperatures where excitation modes are freezing out, or phase changes are imminent. On the "equipartition line" 
note the single point for Cv/k = E/kT = 3/2. If the zero of energy is shifted for this system (e.g. by a phase change or by 
quantum mechanical freezing out of excitation modes), the point stretches into a horizontal line moving between E/kT = 
and E/kT = 3/2. This line approaches its limiting value (here E/kT = 3/2) by moving rightward (as for liquid water in Fig. 
1), or by moving leftward (as for steam in Fig. 1), if the zero of energy is moved, respectively, up or down from its quadratic 
value. 

FIG. 3. Plot of E versus kT (lower left), Cv/k versus E/kT (upper right), and the entropy plots whose slopes associate 
therewith (see text), for a two-state paramagnet consisting of 10 non-interacting spins. This differs from the plots in Fig. 1 
and 2 in that the two left side plots use 1/kT in place of kT because of its more natural mapping of inverted and uninverted 
population states. All quantities are "per spin" values, and of course the energy units So depend on both the spin magnetic 
moment and the magnetic field. Note in particular how the "equipartition line" seems even less relevant here, even though heat 
capacity remains positive as reciprocal temperature dips below zero. For virial systems (like a gravity bound gas), negative 
heat capacities make the concepts of equipartition and degrees-freedom even less apropo, even though the relationships shown 
here remain intact. 
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